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^ ■ Abstract 

Based on the LISSOM model and the OFC earthquake model, we introduce a self-organized 
feature map Neural Network model . It displays a "Self Organized Criticality" (SOC) behavior. 
It can be seen that the feature area (synchronized area)produced by self-organized process brings 
about some definite effect on SOC behavior and the system evolves into a "partly-synchronized" 
^ . state. For explaining this phenomena , a quasi-OFC earthquake model is simulated. 
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I. INTRODUCTION 



A few years ago, Bak, Tang and Wiesenfeld introduced the concept of the "Self- Organized 
Criticality" (SOC) in sand pile model |]J. From then on, this concept has been widely studied 
in some extended dissipative dynamical systems, such as earthquake 0, biology evolution 
||, and so on. It is shown that all these systems can naturally evolve into a "critical state" 
with no intrinsic spatial and temporal scales through a self-organized process without the 
need to fine-tune parameters of the system. This critical state is characterized by a power- 
law distribution of avalanche sizes, where the size is the total number of toppling events or 
unstable units. 

Now,the research on SOC has come into a new level. One has studied several factors' 
influence on SOC behavior, such as network size, periodic or nonperiodic boundary condi- 
tions,local dynamics variable is conservative or not, and so on. Many investigators believe 
that it is intrinsic-stability(order) and variability (disorder)'s common action make the sys- 
tem evolves into a "frozen disorder" (SOC) state [f|, |5], ||. It is the combination feature 
of stability and variability, and it's complex spatial-temporal dynamic behavior, make the 
system in SOC state have maximum complexity and latent computing potency. 

The brain is a complex system and its information process has the properties of stabil- 
ity and variability - on one hand, there are relative stable information stored mechanism, 
and stored area in brain(such as feature area in cortex) ; on the other hand, the brain is 
influenced by the environment and one should continuous update knowledge and concepts. 
The similarity between the SOC systems and the brain has lead us to study Artifical Neu- 
ral Network(ANN) and SOC together. There is some SOC behavior shown in the neuron 
network model introduced by our group 0]. 

The brain is also a complex system with highly complexity, highly order and special 
structure. The structure must have the effect on the brain's dynamics behavior. Our neuron 
network can also produce some special structure, so we believe that the SOC behavior shown 
in our model must have it's own special behavior and rule. 

In this paper, the feature area(produced by self-organized process) 's definite effect on 
SOC behavior has been studied. 
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II. MODEL 



Here we propose a two-dimensional neural network model of square lattice. This model 
is a kind of serial self-organized neural network model, based on the LISSOM model ||. 
When a h- dimensional vector ( is inputted, the state %■(£) of the neuron (i, j) at time t is 
changed according to the formula: 



Vij = a iYl fajfiCh + 7e Yl E ijMVkl(t ~ 1) 
h kl 

-li^2hj,k'i'Vk'i'(t - 1)} 

k'V 

= <r{Mt)} (1) 

where a(x) is an active function, and we design it as sign function, i.e., if x > 0, then 
a(x) = 1 , otherwise a(x) = —1 . is an afferent input weight vector; E^^x is the 

excitatory lateral connection weight on the connection from the neuron (k,l) to the 
neuron; I(ij,kl) is the inhibitory connection weight. fij(t) is the local field of the neuron 
at time t. 7 e and 7, are constant factors. The adjustment of those three connection 
weights is as following according to the dynamic Hebb rule: 

IJ>ij,h(t) + ariijih 



Eij,kl(t + 1) 



Ejj,h(t) + aEVijVki 
Eij [Eij,ki(t) + AEVijVki} ' 



t u hj,k'i'{t) + aiVijVk'i' f . 

where a, oce, oli are the learning rates. 

Note the change of the neuron states is quick and the adjustment of the connection 
weights is slow. Usually, after over 10 iterations of the neuron states when any pattern is 
inputted (at this time, the network state becomes an attractor in the state space, often 
a fixed point.), all connection weights are updated once. Thus, after learning a while, the 
lateral connection weight self-evolves into the" Mexican hat" profile, the afferent input weight 
self-organizes into a topological map of the input space, the neuron network can produce 
some special feature areas, and the state of the neural network is evolved from disordered 
case to stable and topological case in state space. 

Then we introduce the following interactive process between the neurons, similar with 
the pulse coupled interaction: 



1) When the neuron is stable, i.e., = (?{fij(t — 1)} , it doesn't influence the 
others; 

2) If the neuron is unstable, i.e., 77^ 7^ a{fij(t — 1)} or f^t — 1) = , then the 
nearest neighbors (i',f) around this unstable neuron will receive a pulse respectively and 
their local fields will be changed. At the same time, the neuron becomes stable again, 
depending on the formula (1); 

3) When all neurons of the neural network are stable, we choose the minimum g among 
the absolute values of all local fields and drive the local field of every neuron, i.e., 

fij -> fa -c*r ]ij *g (3) 

where c is a constant. 

Now, we present the computer simulation procedure of this model in detail: 

1) Variable initialization. In the 2-dimensional n x n neural network model, let the 
initiatory state and local field equal 0; random initialize each connection weight among 
[—1, 1] ; and produce M random input patterns, Q G [—1, 1], (i — 1, 2, • • • M; j = 1, 2, • • • h). 

2) Learning process. According to formula (1), we input the pattern circularly and iterate 
the neuron state and local field. After period of time, the space state of the network reaches 
stability and we consider the M input patterns have been stored. 

3) Associative memory. Input a new pattern, and then search the unstable neuron 

as defined above in whole neural network. Due to being unstable, the neuron (i, j) discharges 
a pulse to the each nearest neighbor (i', j') and thus causes the local fields of them to change 
as following: 

fi'f -»• fi'f - 7}Vi'j'(l + \fij\) (4) 

where 7 represents the pulse intensity, symbol | | denotes absolute value. 

Simultaneously, according to formula (1), the neuron (i, j) becomes stable as: rjij &(fij) 
, — > rjij ; where a(x) is sign function. 

Repeat this procedure until all neurons of the model are stable. Define one avalanche 
as all unstable neurons in this process. Then begin drive process by formula (3) and new 
avalanche. 
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III. SIMULATION RESULTS 



Recently, Bak and Sneppen have investigated the power law distribution P(X) of the 
distances X between subsequent unstable sites in lattice of BS biology evolution model || , 
and J.De.Boer et al. have found some interested result with it pl.So in this paper we studied 
not only the distribution P(S) of the avalanche sizes S but also the distribution P{X) of the 
distances X between the subsequent unstable sites . We find the distribution will deviate 
from the power law in some conditions. 



A. The effect of synchronized area on SOC behavior 

The size of our lattice is 40 x 40. We find that in associative memory process, the distri- 
bution of the avalanche sizes has power-law behavior, P(S) oc S~ T , r m 0.90 . It is shown 
in Fig.l.a. The distribution P{X) of the distances X between the subsequent unstable sites 
has power-law behavior too, P{X) oc X" 13 , (3 ~ 2.23 , it is shown in Fig.l.b. 

The SOC behavior changed with the scope of lateral connection has been studied. We 
increase the excitatory lateral connection radius d e and the inhibitory connection radius 
di = 3d e . It can be seen in Fig l.a that avalanche size and occurring probability of large 
scale avalanche decrease with the slope d e increasing, and the distribution of P(X) deviates 
from power law more and more in large X, the probability of large distance X between the 
subsequent unstable sites also increases, we consider it is a deviation from SOC behavior, it 
is shown in Fig l.b. 

By investigating , we think our system is in a "partly-synchronized" state, hence the 
dynamics behavior mentioned above can be seen. 

A. Corral et al. propose SOC state and synchronization state might be considered as two 
uttermost state of system (just like two sides of the same coin) ||]. The inhomogeneity 
introduced by boundary or initialization conditions can propagate into interior of network, 



hence makes the system evolve into SOC state [10, 11]. When inhomogeneity is not large 
enough, the system finds a compromise between synchronization and SOC ||. It can be 
considered partly-synchronized state. 

If Our Model is only a pure OFC model without learning process, the system will present 
a macroscopic SOC behavior among almost all the lattices, but it is also a neuron network 
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model. As a kind of self-organized feature map model, after learning a while, it's neurons 
will develop a unique lateral interaction "Mexican hat" profile that represents its long-term 
associations with each other, the afferent input weights will self-organize into a topological 
map of the input space M, it can form some special topological feature regions. We con- 
sider that in these regions, as the connection weights become more topographically ordered, 
neuron's synchronization effect between each other will be reinforced. When order in these 
regions is applied to the model, the system has a tendency from SOC state to synchronized 
state. At last, the system finds a compromise between synchronization and SOC, it could be 
seen as "partly-synchronized" state. With the process, the distribution of avalanche varies 
from the continuous distribution to a discrete one, the possibility of large scale avalanche 
propagating into the interior of the synchronization regions will reduce, and occurring prob- 
ability of large scale avalanche will decrease too, the one-off isolated avalanche (only one 
unstable site in an avalanche)in these regions will increase greatly, it makes the distances 
X between the subsequent unstable sites have a stochastic spatial even distribution in the 
area. This distribution has more effect on probability of large distance between unstable 
sites than probability of small distance. (Because probability of small distance is larger than 
one of large distance.) 

So at this moment, there are some areas in synchronized state and another regions in 
SOC state. We can consider that with the increasing of d e , the feature region(synchronized 
region) produced by self-organized process becomes wider, which results in the whole system 
dynamics deviating largely from SOC state, it can be seen in Fig.l. 

To verify the idea mentioned above, we draw the avalanche's distribution map of the 
whole system. We draw one avalanche's distribution snapshot every 1000 avalanches, then 
overlap all the snapshots in one picture. The result can be seen in Fig2. It can be clearly seen 
that the blank region (seldom avalanche area) expands with the increasing of d e . It means 
that the synchronized region introduced by self-organized expands and large scale avalanches 
reduce more and more. Even though, there are still some isolated unstable neurons in the 
area, it indirectly verifies our deduction of p(X) distribution mentioned above. 

We investigate the relation between average avalanche size (S) and radius d e . From Fig. 3 
, we can see that with the decreasing of d e , (S) will increases, and with d e approaching 0, 
the slope of the curve becomes quite large. This result is approximate to the phenomena 
with the increasing of pulse discharging intensity 7 ]7|], and is consistent with the result in 
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Ref. It implies that the network approaches to SOC state when d e approaching or 7 
approaching 0.5. 

When the ratio T of radius cfj with radius d e is changed , the probability distribution P(S) 
and P(X) and avalanche distribution are changed too, the tendency is similar to Fig.l, Fig.2. 
With the increment of T, the partly-synchronized behavior of system becomes distinctness. 
The phenomena can also be explained by the expanding of synchronized region , but it 
leads us to think that the inhibitory lateral connection and excitatory lateral connection 
have what different effect on synchronized process? It is look like that the inhibitory lateral 
connection has more important effect, but it is not very clear, there are still a lot of work 
to do. 



B. The approximate behavior in a kind of quasi-OFC earthquake model 

OFC earthquake model is a kind of SOC model which has been widely studied in recent 
years @.If we use transformation in our model : 1 — rjijfij —>■ Fij , then formula (4) become 
Fi'ji — > Fi'jt + \Fij . It is the avalanche mechanism of the OFC earthquake model , in fact 
these two models belong to the same class. Therefore we add some synchronized regions in 
OFC model and examine that if the system has the similarity partly-synchronized behavior 
as in our neuron network model. Hence we introduced a cellular automaton model based on 
OFC model, the steps are as follows: 

1) We define a NxN square lattice ,and a LxL square area in the lattice ( L = KN < N) 



2) Initialize all sites to a random value F^ between and 1. 

3) If any F^ > F t h = l,then redistribute the force on F^ to its neighbors according to 
the rule: 

7 I 7 = a < 0.5 , when site (i,j) G L x L region, 

Fi'f -> F Vj i + -F^ I (5) 
z I 7 = 0.5 , otherwise. 

4) Repeat step 3 until no F^ > F th ,we define the avalanche is fully evolved. 

5) Locate the site with the largest strain ,F max ,then drive all sites 

Fij — > Fij + (F th — F max ) (6) 
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and return to step 3. Here we use open boundary conditions ,and serial working mode. It is 
same as our neuron network model but different from traditional OFC model. 

We still focus on the spatial distribution of S and X. Let a = 0.1, then change K, the 
result is shown in Fig 4. It can be seen that the probability distribution P(X) increases and 
deviates from power-law more with K increasing at larger X in Fig 4. a. The tendency is 
similar to that in our previous neuron network model. In Fig 4.b , we can see the occurring 
probability of large scale avalanche reduces with K increasing too, but the phenomena is 
not distinctness as that in previous model. The reason may be that the previous model has 
more complex structure than this model. It produces many pieces of synchronized region, 
but here, we only add a piece of synchronized region in model. 

A. Corral, Grassberger et al. have investigated the influence of pulse discharging intensity 
7 in OFC model. They propose the model present a macroscopic synchronization among 
all the elements of the lattice when 7 is small |4], Therefore in this model , with 

expanding of L x L area(7 = a = 0.1), the synchronized area in the network expands, the 
synchronization behavior becomes distinctness, and the whole system evolves into a partly- 
synchronized state. The dynamics behavior of the previous model is so consistent with this 
model, it suggested that maybe they have the similar dynamics mechanism . Our results 
tend to agree with this idea. 

For studying the partly-synchronized phenomena further, the relation between distribu- 
tion P(x) and a has been investigated. Let K = 0.8 , the partly-synchronized behavior 
become distinctness with a decreasing from 0.5 . But when a less than a number (about 
0.4), there is no more distinctness partly-synchronized behavior introduced by a changing 
(seen in Fig 5) . It can show that the L x L area have evolved into synchronized state. We 
also draw the whole network's avalanches distribution map(Fig .6). It can be clearly seen 
that the isolated avalanches' number inixL area increases with a decreasing, and the large 
scale avalanche can't propagate into the interior of L x L area. These result are consistent 



with the work of C.Tang , Grassberger et al. [10, 13 



IV. CONCLUSION 

In this paper, we analyze the dynamics of the proposed neural network model and find 
the distribution of the avalanche sizes and the distances X between subsequent unstable 
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sites show the power-law behavior. More important, we find the function area and weight 
distribution produced by self-organized process in our Neural Network model will let the 
P(S) and P(X) distribution deviate from power law behavior , and the system evolves into 
a partly-synchronized state at this time. To verify the explanation , we study a quasi-OFC 
earthquake model containing synchronized region , and find it will deviate from power-law 
, evolve into a partly-synchronized state in some conditions too. 

Our self-organized feature map Neuron Network model is just a very simple simulation 
of brain. The real brain has very complex structure and more specific feature regions in 
Cortex. So brain may express a quasi-SOC(partly-synchronized) behavior more than a pure 
SOC behavior. Therefore the stored pattern in brain might be designed as a quasi-SOC 
attractor, and the associate memory process might be designed as the process of the input 
pattern evolving into the attractor. 



Now the neuron synchronization in brain has been observed in many experiments [14 



Rodriguez et al. have investigated the long distance synchronization of human brain activ- 
ity We think it would be interesting to further investigate the relationship between 
synchronization and cognitive , associate process. 
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FIG. 1: The distribution P(S) of the avalanche sizes S, and the distribution P(X) of distances 
X between subsequent unstable sites, changed with lateral connection radius d e , for our 40 x 40 
neuron network model. Where d{ = 2d e , 7 = 0.5. (a). Log- log plot of P(S) .vs. sizes S. (b). 
Log-log plot of P(X) .vs. distances X. 

FIG. 2: The avalanche distribution map for our 40 x 40 neuron network model , where d{ = 2d e , 
7 = 0.5. (o) represent the origin sites of avalanche. shadow(+) represent the avalanche region. 
(a)-(d) correspond to d e = 1,2,4,6 

FIG. 4: The distribution P(X) of the distance X between subsequent unstable sites and the 
distribution P(S) of the avalanche sizes S , changed with K = L/N. for our 40 x 40 quasi-OFC 
earthquake model. Where in L x L area,7 = a = 0.1, out of this area, 7 = 0.5. (a). Log-log plot 
of P(X) .vs. distances X . (b). Log-log plot of P(S) .vs. sizes S. 



FIG. 5: The distribution P(X) of the distance X between subsequent unstable sites and the 
distribution P(S) of the avalanche sizes S , changed with pulse discharging intensity a in L x L 
area for our 40 x 40 quasi-OFC earthquake model. Where K = L/N = 0.8, out of L x L area 
7 = 0.5. (a). Log-log plot of P(X) .vs. distances X. (b). Log-log plot of P(S) .vs. sizes S. 



FIG. 6: The avalanche distribution map for our 40 x 40 quasi-OFC earthquake model , where 
K = L/N = 0.8, out of L x L area 7 = 0.5. pulse discharing intersity a is changed, (o) represent the 
origin sites of avalanche. shadow(+) represent the avalanche region. The map(a)-(d) correspond 
to a = 0.5,0.48,0.4,0.1 

FIG. 3: The avalanche average size (S) as a function of the lateral connection radius d e for our 
40 x 40 neuron network model , where di = 2d e , 7 = 0.5. With decreasing of d e , (S) increases,and 
with d e approaching 0, the slope of the curve become quite large. 
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